Skip to content

Add first-class adversarial fuzz runtime contracts - #2020

Merged
chubes4 merged 1 commit into
mainfrom
feat/2014-adversarial-fuzzing
Jul 24, 2026
Merged

Add first-class adversarial fuzz runtime contracts#2020
chubes4 merged 1 commit into
mainfrom
feat/2014-adversarial-fuzzing

Conversation

@chubes4

@chubes4 chubes4 commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a deterministic, stateful adversarial campaign engine with corpus retention, generic mutations, bounded concurrent workers, novelty signals, minimization, stable finding fingerprints, differential classification, and replay schedules
  • add a typed transport fault model, truthful Playwright fault and clock adapters, bounded sealed evidence bundles, and disruptable Redis/SMTP/HTTP/MySQL service providers
  • publish the contracts through the stable facades and runtime manifest, preserve existing fuzz/MySQL behavior, and document architecture, fidelity, safety, compatibility, and residual boundaries

Architecture

runtime-core owns backend-neutral contracts and algorithms only:

  • adversarial-campaign.ts: deterministic round-based scheduling, stateful corpus mutation, novelty retention, resource budgets, minimization, deduplication, replay, and differential classification
  • transport-faults.ts: request matching, deterministic outcome sequences/counters, capability negotiation, evidence redaction, and fault fingerprints
  • adversarial-browser.ts: descriptor-derived hostile journeys, generic UI oracles, and journey minimization
  • adversarial-artifacts.ts: bounded manifested findings/replays, secret/path redaction, file hashes, and a stable bundle digest

runtime-playground owns Playwright mappings for browser routing and browser clock control. cli owns Docker-backed disposable service lifecycle and disruption controls. No component, plugin, product, transport vendor, or WordPress-specific attack policy enters the generic core.

Public contracts

New discoverable schemas:

  • wp-codebox/adversarial-campaign/v1
  • wp-codebox/adversarial-campaign-result/v1
  • wp-codebox/adversarial-finding/v1
  • wp-codebox/adversarial-replay/v1
  • wp-codebox/adversarial-evidence-bundle/v1
  • wp-codebox/transport-fault-model/v1
  • wp-codebox/transport-fault-capabilities/v1
  • wp-codebox/transport-fault-evidence/v1
  • wp-codebox/adversarial-browser-plan/v1
  • wp-codebox/adversarial-browser-oracle-result/v1
  • wp-codebox/clock-control-capabilities/v1
  • wp-codebox/differential-result/v1

They are exported from the compatibility, curated public, and contracts facades and listed by runtimeContractManifest().

Determinism and concurrency

Cases are generated from a seeded SHA-256 schedule. Up to workers cases execute concurrently in each round, then results are committed to the retained corpus in case-index order. Completion timing therefore cannot alter later mutations. Tests prove three workers overlap while two runs retain identical schedules, corpora, fingerprints, and replay metadata.

Findings are minimized by deterministic action-subset replay plus input shrinking. Fingerprints intentionally exclude payload-specific noise and include oracle ids, normalized diagnostic codes, status, and matrix cell for stable deduplication.

Capability and fidelity model

Every transport adapter negotiates required semantics before execution:

  • exact in Playwright: response substitution, delay, seeded jitter, host remap
  • explicitly emulated: malformed/truncated payloads, bandwidth delay, timeout, refusal/reset, request/response byte corruption
  • explicitly unsupported: HTTP chunk framing, socket half-close, disconnect-after-N-bytes

Playwright payload truncation is never mislabeled as a transport disconnect. Browser clock freeze/advance/skew/resume is exact; server process, scheduler, and database clocks are explicitly unsupported by that adapter.

Services

inputs.services additively accepts:

  • Redis with persistence disabled
  • SMTP message sink with a loopback inspection port
  • deterministic HTTP response fixture
  • existing MySQL/MariaDB behavior

All services use ephemeral loopback ports, tmpfs/no persistent volume, explicit reverse-order teardown, and runtime-created credentials where required. Typed controls support stop/start, pause/resume, restart, disconnect/reconnect, and provider-specific flush/read-only behavior. Recovery waits for readiness. Unsupported latency shaping points callers to the transport-fault adapter instead of pretending fidelity.

Security boundaries

  • campaign budgets bound cases, actions, input bytes, case time, wall time, workers, and artifact bytes
  • evidence redacts sensitive headers/fields, caller-declared values, credentials, URL secrets, and machine-specific paths
  • fault evidence records request/fault/response metadata without serializing secrets
  • services bind only to loopback and carry no persistent volumes
  • existing Codebox network-deny, mount, process, CPU/memory/disk, disposable sandbox, checkpoint, and teardown boundaries remain authoritative

Compatibility

  • existing recipes and authored fuzzRun.cases are unchanged
  • MySQL/MariaDB configuration, outputs, authentication, readiness, and explicit teardown remain supported
  • new service kinds and TypeScript contracts are additive
  • no version or changelog was edited

Artifact and replay example

Each retained finding carries:

seed + corpus id + worker/iteration + matrix cell
minimized actions/input + normalized worker schedule + fault schedule
exact declared provenance + stable SHA-256 fingerprint
wp-codebox adversarial replay --campaign <id> --case <id>

writeAdversarialEvidenceBundle() writes manifest.json, the campaign result, fingerprint-addressed finding and replay files, and evidence/secret-scan.json, enforcing the declared artifact byte ceiling before publication.

Test matrix

Passing:

  • npm run test:adversarial-runtime (11 contract/unit/integration tests)
  • npm run test:runtime-services
  • npm run test:runtime-services-lifecycle
  • npm run test:fuzz-suite-runner
  • npm run test:playground-fuzz-suite-public
  • npm run test:browser-scenarios
  • npm run test:schema-parity
  • npm run test:public-api-contract
  • npm run test:runtime-contract-manifest
  • npm run test:release-package-coverage
  • npm run build
  • npm pack --dry-run --json
  • git diff --check

The repository-wide npm run check reaches the pre-existing command-registry-smoke failure tracked in #1745: wordpress.collect-workload-result outputShape should mention outputSchema id. This branch does not touch that command definition.

Performance and resource limits

Defaults are 100 cases, one worker, 50 actions/case, 1 MiB input/case, 30 seconds/case, five minutes/campaign, and 100 MiB artifacts. Values are normalized to hard ceilings (100,000 cases, 128 workers, 10,000 actions, 64 MiB input, one hour/case, one day/campaign, and 4 GiB artifacts). Adapters remain responsible for lower backend limits.

Known residual gaps

This is the strongest coherent generic substrate slice, but it does not claim the complete #2014 acceptance criteria yet:

The current neutral tests prove deterministic discovery/minimization behavior without pretending they are live WordPress plugin/theme campaigns. For that reason this PR deliberately uses Refs #2014, not Closes #2014.

Refs #2014

@chubes4
chubes4 merged commit e8de154 into main Jul 24, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant